{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 手写字母XO数据预处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 概要"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "先尝试做一下分类，　通过卷积神经网络分类，然后完成Tic-Tac-Toe的游戏．\n",
    "\n",
    "数据集可以来自于手写字母，提取字母中的`X`跟`O` ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EMNIST数据集\n",
    "\n",
    "https://www.nist.gov/node/1298471/emnist-dataset\n",
    "\n",
    "需要其中的**EMNIST Letters: 145,600 characters. 26 balanced classes.**数据集\n",
    "\n",
    "下载里面的gzip压缩包，只保留其中的`emnist-letters` 部分，手写字母数据集.\n",
    "\n",
    "解压缩里面的`gz`后缀的压缩包，里面是二进制格式的训练集跟测试集以及标签数据．\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![emnist-letters.png](./image/emnist-letters.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "安装拓展包`mlxtend` \n",
    "\n",
    "```\n",
    "sudo pip3 install mlxtend\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 载入数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlxtend.data import loadlocal_mnist\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# 标签数据的二进制文件路径\n",
    "LABEL_FILE = \"../../dataset/emnist-letters/emnist-letters-train-labels-idx1-ubyte\";\n",
    "# 图像数据的二进制文件路径\n",
    "IMAGE_FILE = \"../../dataset/emnist-letters/emnist-letters-train-images-idx3-ubyte\";\n",
    "# 载入训练集与标签\n",
    "X, y = loadlocal_mnist(images_path=IMAGE_FILE,labels_path=LABEL_FILE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "X是一个长度为124800x784的矩阵，每一行就是一张图片对应的向量, 维度是(784,)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(124800, 784)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 显示样例数据集"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果你想可视化图像需要将向量转换为28x28的矩阵，接下来再把矩阵转置一下．"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "另外标签的范围是1-26, 代表`a`-`z` 26个字母．"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "z\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAADohJREFUeJzt3X+IVfeZx/HP468gtmhcXZHUqFOTBaOsXQZZ3Ji4NDFuCJrmj1BJFjdIp39U2EIDG9zA+k9AlrRNSaBkpFoj3aihDREiu82aDdNAU2J+rEazbWbDmCo6E+OPWklwR5/9Y45lYuZ8z829595zJs/7BTL3nud+5z5c/My5937POV9zdwGIZ0LVDQCoBuEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxDUpE4+mZlxOCHQZu5ujTyupT2/ma0xs9+aWb+ZPdLK7wLQWdbssf1mNlHS7yTdKem4pNclrXf3o4kx7PmBNuvEnn+5pH53f9/dL0naLWldC78PQAe1Ev4bJP1+1P3j2bZPMbMeMztoZgdbeC4AJWv7F37u3iupV+JtP1Anrez5T0iaN+r+V7JtAMaBVsL/uqSbzGyhmU2R9E1J+8ppC0C7Nf22392HzWyTpP+QNFHSdnc/UlpnKMWkSa19shseHi6pE9RN01N9TT0Zn/k7jvDH05GDfACMX4QfCIrwA0ERfiAowg8ERfiBoDp6Pj/aY9asWbm19evXJ8fOmDEjWd+/f3+yfv78+WS9SufOncutnT59uoOd1BN7fiAowg8ERfiBoAg/EBThB4Ii/EBQnNVXA2bpk7DWrl2brD/00EO5tbvuuis59rrrrkvWL1++nKxfuXKlpfEpEyak901Fr9uxY8dyaytWrEiOHc9TgZzVByCJ8ANBEX4gKMIPBEX4gaAIPxAU4QeCYp6/BlKn5ErSq6++mqzPnz8/t1Y0V150Su7Zs2eT9QsXLiTrr7zySm7t4sWLybFLlixJ1leuXJmsT58+Pbd2yy23JMf29/cn63XGPD+AJMIPBEX4gaAIPxAU4QeCIvxAUIQfCKqlS3eb2YCkC5IuSxp29+4ymopm2rRpyfpbb72VrO/Zsye3VnQ+/eHDh1uqF63iOzg42PTYouMfNm/enKxv3LgxWY+ujOv2/627j98rHwBB8bYfCKrV8LukX5rZG2bWU0ZDADqj1bf9t7r7CTP7c0kvmdn/uHvf6AdkfxT4wwDUTEt7fnc/kf0ckvS8pOVjPKbX3bv5MhCol6bDb2bTzOzLV29LWi3pnbIaA9BerbztnyPp+ezyyZMk/Zu7/3spXQFou6bD7+7vS/rLEnsJK3V9eUl64IEHkvWia+ePV7Nnz07Wb7vttmQ9dYxD0TEGETDVBwRF+IGgCD8QFOEHgiL8QFCEHwiqjLP60GZf1Km8SZPS//1Wr16drHd1dSXrqctvnzp1Kjk2Avb8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU8/yozIIFC5L1oiW4i5YHf+KJJ3Jrn3zySXJsBOz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAo5vnRVlOnTs2tPffcc8mxixYtStYff/zxZH3v3r3JenTs+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gqMJ5fjPbLukeSUPuviTbNlPSHkkLJA1Iut/dz7avTdSVmSXrN998c26taB7/5ZdfTtZ37dqVrH/88cfJenSN7Pl/KmnNNdsekXTA3W+SdCC7D2AcKQy/u/dJOnPN5nWSdma3d0q6t+S+ALRZs5/557j7yez2KUlzSuoHQIe0fGy/u7uZeV7dzHok9bT6PADK1eyef9DM5kpS9nMo74Hu3uvu3e7e3eRzAWiDZsO/T9KG7PYGSS+U0w6ATikMv5k9K+nXkv7CzI6b2UZJWyXdaWbvSbojuw9gHCn8zO/u63NKXy+5F9RQ0Tx+V1dXsv7www83/dx9fX3J+gcffND07wZH+AFhEX4gKMIPBEX4gaAIPxAU4QeC4tLdSFq7dm2yXnT57Pnz5+fWenrSR33v2bMnWR8eHk7WkcaeHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCYp4/uEmT0v8FHnzwwWQ9NY8vSceOHcut7d+/PzmWS2+3F3t+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiKef4vuIkTJybrt99+e0v1M2euXcP10x577LHc2ocffpgci/Zizw8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRXO85vZdkn3SBpy9yXZti2SviXp6kTtZndPn5yNtkkto71w4cLk2E2bNiXr06ZNS9b37t2brKfO2Xf35Fi0VyN7/p9KWjPG9h+6+7LsH8EHxpnC8Lt7n6T0YVwAxp1WPvNvMrNDZrbdzK4vrSMAHdFs+H8s6auSlkk6Ken7eQ80sx4zO2hmB5t8LgBt0FT43X3Q3S+7+xVJ2yQtTzy219273b272SYBlK+p8JvZ3FF3vyHpnXLaAdApjUz1PStplaRZZnZc0r9IWmVmyyS5pAFJ325jjwDawDo512pmTOw2oeic/KVLl+bWduzYkRy7ePHiZL2npydZL5rn59r7nefu+Qd+jMIRfkBQhB8IivADQRF+ICjCDwRF+IGguHT3OLBq1apkPXVa7qJFi5Jj+/r6kvUXX3wxWWcqb/xizw8ERfiBoAg/EBThB4Ii/EBQhB8IivADQTHPXwNTp05N1osur33HHXfk1vr7+5Njn3rqqWT99OnTyTrGL/b8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU8/wlSC2RLUldXV3J+pYtW5L1FStWJOu7d+/OrT366KPJsYODg8k6vrjY8wNBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIXz/GY2T9IzkuZIckm97v4jM5spaY+kBZIGJN3v7mfb12p9zZ49O1kvmmu/7777kvVt27Yl61u3bs2tDQ0NJccirkb2/MOSvufuiyX9taTvmNliSY9IOuDuN0k6kN0HME4Uht/dT7r7m9ntC5LelXSDpHWSdmYP2ynp3nY1CaB8n+szv5ktkPQ1Sb+RNMfdT2alUxr5WABgnGj42H4z+5Kkn0v6rrv/YfTx7O7uZuY543ok9bTaKIByNbTnN7PJGgn+z9z9F9nmQTObm9XnShrzmyV373X3bnfvLqNhAOUoDL+N7OJ/Iuldd//BqNI+SRuy2xskvVB+ewDapZG3/X8j6e8lHTazt7NtmyVtlbTXzDZKOibp/va0WH8rV65M1tesWZOsX7x4MVk/cOBAsp6aznMf89MYUBx+d39VUt4J618vtx0AncIRfkBQhB8IivADQRF+ICjCDwRF+IGgrJPzwHmHAI8Hs2bNyq299tprybE33nhjsv70008n608++WSyfuXKlWS9FRMnTkzWFy9enKwvXbo0t1Z0yfNWpV6XXbt2JccODAyU3E3nuHtDLyx7fiAowg8ERfiBoAg/EBThB4Ii/EBQhB8Iinn+Bi1atCi3duTIkeTYyZMnJ+tnzpxJ1j/66KNkvZ2K5uKnT5+erM+YMaPp392q1Dz/pk2bkmN37NiRrF++fLmpnjqBeX4ASYQfCIrwA0ERfiAowg8ERfiBoAg/EFTDy3VFNzw8nFu7dOlScuyUKVOS9ZkzZybrqblyqb3n8589m151/fz588n6uXPnymznc0nNxRddg6HO8/hlYc8PBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0EVns9vZvMkPSNpjiSX1OvuPzKzLZK+JenD7KGb3X1/we8at+fzp849v+eee5Jjly1blqxPmJD+G3zo0KFk/ejRo7m1VuerW53H7+T1Ij6P1HEb412j5/M3cpDPsKTvufubZvZlSW+Y2UtZ7Yfu/nizTQKoTmH43f2kpJPZ7Qtm9q6kG9rdGID2+lyf+c1sgaSvSfpNtmmTmR0ys+1mdn3OmB4zO2hmB1vqFECpGg6/mX1J0s8lfdfd/yDpx5K+KmmZRt4ZfH+sce7e6+7d7t5dQr8AStJQ+M1sskaC/zN3/4Ukufugu1929yuStkla3r42AZStMPw28jX3TyS96+4/GLV97qiHfUPSO+W3B6BdGpnqu1XSryQdlnT13NHNktZr5C2/SxqQ9O3sy8HU76rnvE+LiqbqiupFik7ZbecpvRh/Gp3q47r9JSD8qBOu2w8gifADQRF+ICjCDwRF+IGgCD8QFFN9wBcMU30Akgg/EBThB4Ii/EBQhB8IivADQRF+IKhOL9F9WtKxUfdnZdvqqK691bUvid6aVWZv8xt9YEcP8vnMk5sdrOu1/eraW137kuitWVX1xtt+ICjCDwRVdfh7K37+lLr2Vte+JHprViW9VfqZH0B1qt7zA6hIJeE3szVm9lsz6zezR6roIY+ZDZjZYTN7u+olxrJl0IbM7J1R22aa2Utm9l72c8xl0irqbYuZncheu7fN7O6KeptnZv9lZkfN7IiZ/WO2vdLXLtFXJa9bx9/2m9lESb+TdKek45Jel7Te3fPXme4gMxuQ1O3ulc8Jm9ltkv4o6Rl3X5Jt+1dJZ9x9a/aH83p3/6ea9LZF0h+rXrk5W1Bm7uiVpSXdK+kfVOFrl+jrflXwulWx518uqd/d33f3S5J2S1pXQR+15+59ks5cs3mdpJ3Z7Z0a+c/TcTm91YK7n3T3N7PbFyRdXVm60tcu0Vclqgj/DZJ+P+r+cdVryW+X9Esze8PMeqpuZgxzRq2MdErSnCqbGUPhys2ddM3K0rV57ZpZ8bpsfOH3Wbe6+19J+jtJ38ne3taSj3xmq9N0TUMrN3fKGCtL/0mVr12zK16XrYrwn5A0b9T9r2TbasHdT2Q/hyQ9r/qtPjx4dZHU7OdQxf38SZ1Wbh5rZWnV4LWr04rXVYT/dUk3mdlCM5si6ZuS9lXQx2eY2bTsixiZ2TRJq1W/1Yf3SdqQ3d4g6YUKe/mUuqzcnLeytCp+7Wq34rW7d/yfpLs18o3//0r65yp6yOmrS9J/Z/+OVN2bpGc18jbw/zTy3chGSX8m6YCk9yT9p6SZNeptl0ZWcz6kkaDNrai3WzXylv6QpLezf3dX/dol+qrkdeMIPyAovvADgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxDU/wPaEdR01wm5JwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def array2img(x):\n",
    "    '''将(784,)的向量转换为图片格式28*28'''\n",
    "    x_img = x.reshape((28, 28))\n",
    "    x_img = x_img.T\n",
    "    return x_img\n",
    "\n",
    "def num2letter(num):\n",
    "    '''将标签数字转换为字母'''\n",
    "    return chr(ord('a') + num - 1)\n",
    "\n",
    "# 显示图片\n",
    "plt.imshow(array2img(X[14]), cmap='gray')\n",
    "print(num2letter(y[14]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 筛选数据\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "去除无关字符的测试集, 只保留`X`跟`O`对应的数据集还有标签，并将标签替换为0跟1\n",
    "\n",
    "| 字母 | 标签 |\n",
    "| ---- | ---- |\n",
    "| O    | 0    |\n",
    "| X    | 1    |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def letter2label(letter):\n",
    "    return ord(letter) - ord('a') + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 获取字母X跟O对应的序号\n",
    "xo_indexs = np.where(np.logical_or(y==letter2label('o'), y==letter2label('x')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([     3,      9,     17, ..., 124742, 124748, 124798]),)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xo_indexs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 索引数据\n",
    "X_train = X[xo_indexs]\n",
    "y_train = y[xo_indexs]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 标签转换为0,1\n",
    "y_train = np.uint8(y_train == letter2label('x'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 转换为2D数据集\n",
    "将原来的1维的数据重新转换为二维"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "image_list = []\n",
    "for img_idx in range(len(X_train)):\n",
    "    image_list.append(array2img(X_train[img_idx]).tolist())\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 训练集图片\n",
    "train_images = np.array(image_list).astype('uint8')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f6c1ee9dda0>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAENBJREFUeJzt3WuMlFWex/HfX9BWQa6T7SCD2+NETQTiJUSJdsTV1XgN6AuCLwyTmQzzYjBOsi/WuIlrXDcZNzuzmVeTMNEMbGbFTQA1akZmySqsMUQwLNcdRG0dkMsIAg2R+39f9MNui/38T1G3p5rz/SSkq+rfp+pQ3b9+quo85xxzdwHIz0VVdwBANQg/kCnCD2SK8AOZIvxApgg/kCnCD2SK8AOZIvxApka288HMjNMJgRZzd6vl+xo68pvZfWb2RzPbYWZPNXJfANrL6j2338xGSNou6R5JOyV9IOkxd98atOHID7RYO478t0ja4e6fuPsJSUslzW7g/gC0USPhnyzpT4Ou7yxu+wYzW2Bm68xsXQOPBaDJWv6Bn7svkrRI4mU/0EkaOfLvkjRl0PXvFrcBGAYaCf8Hkq4xs++Z2SWS5kl6vTndAtBqdb/sd/dTZrZQ0tuSRkh6yd23NK1nAFqq7qG+uh6M9/xAy7XlJB8AwxfhBzJF+IFMEX4gU4QfyBThBzLV1vn8ubroovhvbKp+oTpz5kxDdTQmz986AIQfyBXhBzJF+IFMEX4gU4QfyBRDfTWKhuO6urrCtqNHjw7rY8eOratPw8GpU6dKa0ePHg3bfvXVV3XfN9I48gOZIvxApgg/kCnCD2SK8AOZIvxApgg/kCnG+QsTJkwI69OnTy+tPfjgg2Hbq6++OqxPnTo1rA/nKb/9/f2ltb6+vrDt6tWrw/rSpUvD+oEDB0prnCPAkR/IFuEHMkX4gUwRfiBThB/IFOEHMkX4gUw1NM5vZn2S+iWdlnTK3Wc0o1NVGDNmTFi/4YYbSmtz5swJ26bm648bNy6sm9W06WpHOnnyZGltypQpYduJEyeG9fXr14f1HTt2lNb27dsXtm3n7tVVacZJPn/l7l824X4AtBEv+4FMNRp+l7TSzNab2YJmdAhAezT6sr/X3XeZ2V9I+oOZ/Y+7f+OE7OKPAn8YgA7T0JHf3XcVX/dJWiHpliG+Z5G7zxjOHwYCF6K6w29mo8zsirOXJd0raXOzOgagtRp52d8taUUxDDVS0r+5+++b0isALWftHM80s8oGTy+55JKwPm/evLD+6KOPltYeeuihsG1qPn5qHL+Rn9Hp06dbdt+1iP7vqeclOkdAkrZu3RrWN2zYUFp7/vnnw7ZffPFFWP/666/DepXcvaYTQxjqAzJF+IFMEX4gU4QfyBThBzJF+IFMZbN0d2ra7MMPPxzWZ86cWVprdGntY8eOhfW9e/fW3f7tt98O20bLW9ci9X+PliXv6ekJ21511VVhPVpOXZKuvfba0trIkfGv/ltvvRXWly9fHtaPHz8e1jsBR34gU4QfyBThBzJF+IFMEX4gU4QfyBThBzLFOH9h/PjxYT01JTiSmjabmj66cuXKsB5tN71s2bKGHjtlxIgRYf36668vraW2Lr/jjjvC+l133RXWu7q6Smu33npr2Da1hfeqVavCempp8E7AkR/IFOEHMkX4gUwRfiBThB/IFOEHMkX4gUxdMEt3p+ZnP/HEE2H9ueeeC+uXX355ae3EiRNh29RY+jPPPBPW33jjjbAeLf199OjRsG1qeexGNbJ0d2o+/+OPPx7Wb7rpptLa3XffHbY9cuRIWF+4cGFYf/XVV8N6akn1RrB0N4AQ4QcyRfiBTBF+IFOEH8gU4QcyRfiBTCXn85vZS5IekrTP3acVt02Q9IqkHkl9kua6+1et62ZaapvrMWPGhPXUfP3o/g8dOhS23bhxY1hfu3ZtWO/v7w/r0bka7TyPYyhnzpypqyZJu3btCuuvvfZa3e1T8/lHjx4d1qdNmxbW33zzzbDeynH+WtVy5P+tpPvOue0pSavc/RpJq4rrAIaRZPjdfbWkc7d1mS1pcXF5saQ5Te4XgBar9z1/t7vvLi7vkdTdpP4AaJOG1/Bzd4/O2TezBZIWNPo4AJqr3iP/XjObJEnF19LVCt19kbvPcPcZdT4WgBaoN/yvS5pfXJ4vKf7YFUDHSYbfzF6W9L6k68xsp5n9SNLPJd1jZh9J+uviOoBhJPme390fKynFE6JbIJr/nVqXP7WXe2r9+Whcds2aNWHbpUuXhvVPP/00rKfGwy9UqT3uN23aFNajdRTuu+/c0etv6u3tDeuPPPJIWF+xYkVY3759e2nt2LFjYdtm4Qw/IFOEH8gU4QcyRfiBTBF+IFOEH8jUsNqiOxqOS22xnZqCmXLgwLlzm/7fK6+8ErZNDQV2wvTO4Sj1vEU/s/fffz9s29PTE9avu+66sJ5aGvzw4cOltb6+vrBts3DkBzJF+IFMEX4gU4QfyBThBzJF+IFMEX4gU8NqnD/ahju11HJqae7UtNloq+vUlNzU0ttojehn+sknn4RtU2PtU6dODeup38fUlvLtwJEfyBThBzJF+IFMEX4gU4QfyBThBzJF+IFMVT/YeB66u8u3BJw1a1bdbSXp5MmTYT2aG75///6G7hutEY3zb926NWy7ZcuWsH7//feH9WiZ+U7R+T0E0BKEH8gU4QcyRfiBTBF+IFOEH8gU4QcylRznN7OXJD0kaZ+7Tytue1bSjyX9ufi2p939rVZ18qxG5vOn5k8fPHgwrEfzu6O5/lK+W2x3stSa/zn8zGo58v9W0lCbmf+Lu99Y/Gt58AE0VzL87r5aUvnpbQCGpUbe8y80s41m9pKZxXtlAeg49Yb/15K+L+lGSbsl/aLsG81sgZmtM7N1dT4WgBaoK/zuvtfdT7v7GUm/kXRL8L2L3H2Gu8+ot5MAmq+u8JvZpEFXH5G0uTndAdAutQz1vSzpTknfMbOdkv5e0p1mdqMkl9Qn6Sct7COAFkiG390fG+LmF1vQl4ak5k+bWVg/dOhQWN+2bVtpLdprXZLcPayjNaKf+bhx48K2Y8aMqfu+hwvO8AMyRfiBTBF+IFOEH8gU4QcyRfiBTA2rpbtPnTpVWksNt6WWz54wYUJYv+2220prqWWcN27cGNY///zzsB79vztdNATb6PLWXV1dYX3SpEmltSeffDJsG/28pXTfh8OUYI78QKYIP5Apwg9kivADmSL8QKYIP5Apwg9kaliN80dLZEdLa0tSf39/WB8/Pl6GcPr06aW12bNnh23Hjh0b1tesWRPWd+/eHdaj8wBaPZ04NbU1mjo7atSosG1qufXUtNxp06aV1mbOnBm2jc4RkNJLfx85ciSsd8K5Gxz5gUwRfiBThB/IFOEHMkX4gUwRfiBThB/IlLVzWWkza+jBRowYUVqbOHFi2PaFF14I63Pnzg3rl156aWntxIkTYdvUOQap+fzvvPNOWI/Of2j1vPLUvPapU6eW1np6esK2V1xxRVi/+OKLw3q0bXvq9yUltUbD/Pnzw/r27dtLa8eOHaurT2e5e03rinPkBzJF+IFMEX4gU4QfyBThBzJF+IFMEX4gU8n5/GY2RdISSd2SXNIid/+VmU2Q9IqkHkl9kua6+1et62o8hzq1xfbKlSvDem9vb1i/8sorS2uXXXZZ2DY1Hp2a157aUyC1J0GVojn30Ti8lH7eUqJzEFLnP6TG2jds2BDW9+zZE9Y74WdWy5H/lKS/cffrJc2U9FMzu17SU5JWufs1klYV1wEME8nwu/tud/+wuNwvaZukyZJmS1pcfNtiSXNa1UkAzXde7/nNrEfSTZLWSup297PrS+3RwNsCAMNEzWv4mdloScsk/czdDw9eu83dvey8fTNbIGlBox0F0Fw1HfnN7GINBP937r68uHmvmU0q6pMk7RuqrbsvcvcZ7j6jGR0G0BzJ8NvAIf5FSdvc/ZeDSq9LOjt1ab6k15rfPQCtkpzSa2a9ktZI2iTp7PjI0xp43//vkq6S9JkGhvoOJO6rffOHz5EajpszJ/688t577y2tpbbobnRqajSVWUovn92pUsNtqXpquCyaSp1aLn3Tpk1hfcmSJWH9s88+C+utnEpf65Te5Ht+d/8vSWV3dvf5dApA5+AMPyBThB/IFOEHMkX4gUwRfiBThB/I1LBauruVurq6wnq0zfbtt98etk0tUT158uSwfuedd4b1aEpwamntlNQ5CKnpxtHW6R9//HHYdvPmzWH94MGDdT/2e++9F7Y9fPhwWG90ee1WYuluACHCD2SK8AOZIvxApgg/kCnCD2SK8AOZYpy/RtGc+tRYd6NLc8+aNSusR0tgp8b5U/VU32+++eaw/u6775bWtmzZErZNzak/fvx4WI+2Lj9wIFx6IlwmvtMxzg8gRPiBTBF+IFOEH8gU4QcyRfiBTBF+IFOM8w8DI0fWvKvaeUut+Z967NQ5Cvv37y+tpdbdH85j7VVinB9AiPADmSL8QKYIP5Apwg9kivADmSL8QKaS4/xmNkXSEkndklzSInf/lZk9K+nHkv5cfOvT7v5W4r4Y57/AROscSIzVV6HWcf5awj9J0iR3/9DMrpC0XtIcSXMlHXH3f661U4T/wkP4O0+t4U+eOubuuyXtLi73m9k2SfEWMwA63nm95zezHkk3SVpb3LTQzDaa2UtmNr6kzQIzW2dm6xrqKYCmqvncfjMbLeldSf/o7svNrFvSlxr4HOAfNPDW4IeJ++Bl/wWGl/2dp2nv+SXJzC6W9Iakt939l0PUeyS94e7TEvdD+C8whL/zNG1ijw1M+3pR0rbBwS8+CDzrEUnxlqoAOkotn/b3SlojaZOkM8XNT0t6TNKNGnjZ3yfpJ8WHg9F9ceQHWqypL/ubhfADrcd8fgAhwg9kivADmSL8QKYIP5Apwg9kivADmSL8QKYIP5Apwg9kivADmSL8QKYIP5Apwg9kqnV7Pw/tS0mfDbr+neK2TtSpfevUfkn0rV7N7Ntf1vqNbZ3P/60HN1vn7jMq60CgU/vWqf2S6Fu9quobL/uBTBF+IFNVh39RxY8f6dS+dWq/JPpWr0r6Vul7fgDVqfrID6AilYTfzO4zsz+a2Q4ze6qKPpQxsz4z22RmG6reYqzYBm2fmW0edNsEM/uDmX1UfB1ym7SK+vasme0qnrsNZvZARX2bYmb/aWZbzWyLmT1Z3F7pcxf0q5Lnre0v+81shKTtku6RtFPSB5Iec/etbe1ICTPrkzTD3SsfEzazOyQdkbTk7G5IZvZPkg64+8+LP5zj3f1vO6Rvz+o8d25uUd/Kdpb+gSp87pq543UzVHHkv0XSDnf/xN1PSFoqaXYF/eh47r5a0oFzbp4taXFxebEGfnnarqRvHcHdd7v7h8Xlfklnd5au9LkL+lWJKsI/WdKfBl3fqc7a8tslrTSz9Wa2oOrODKF70M5IeyR1V9mZISR3bm6nc3aW7pjnrp4dr5uND/y+rdfdb5Z0v6SfFi9vO5IPvGfrpOGaX0v6vga2cdst6RdVdqbYWXqZpJ+5++HBtSqfuyH6VcnzVkX4d0maMuj6d4vbOoK77yq+7pO0QgNvUzrJ3rObpBZf91Xcn//j7nvd/bS7n5H0G1X43BU7Sy+T9Dt3X17cXPlzN1S/qnreqgj/B5KuMbPvmdklkuZJer2CfnyLmY0qPoiRmY2SdK86b/fh1yXNLy7Pl/RahX35hk7ZublsZ2lV/Nx13I7X7t72f5Ie0MAn/h9L+rsq+lDSr6sl/Xfxb0vVfZP0sgZeBp7UwGcjP5I0UdIqSR9J+g9JEzqob/+qgd2cN2ogaJMq6luvBl7Sb5S0ofj3QNXPXdCvSp43zvADMsUHfkCmCD+QKcIPZIrwA5ki/ECmCD+QKcIPZIrwA5n6X2CltpIE5lVcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 显示其中一个样例图片\n",
    "plt.imshow(train_images[0], cmap='gray')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 训练集的标签\n",
    "train_labels = y_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 测试集执行相同操作\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_image_file = '../../dataset/emnist-letters/emnist-letters-test-images-idx3-ubyte'\n",
    "test_label_file = '../../dataset/emnist-letters/emnist-letters-test-labels-idx1-ubyte'\n",
    "X_test, y_test = loadlocal_mnist(images_path=test_image_file,labels_path=test_label_file)\n",
    "xo_indexs = np.where(np.logical_or(y_test==letter2label('o'), y_test==letter2label('x')))\n",
    "X_test = X_test[xo_indexs]\n",
    "y_test = y_test[xo_indexs]\n",
    "y_test = np.uint8(y_test == letter2label('x'))\n",
    "image_list = []\n",
    "for img_idx in range(len(X_test)):\n",
    "    image_list.append(array2img(X_test[img_idx]).tolist())\n",
    "test_images = np.array(image_list).astype('uint8')\n",
    "test_labels = y_test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 序列化保存数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 创建一个字典格式的数据集\n",
    "dataset = {\n",
    "    'train_images': train_images,\n",
    "    'train_labels': train_labels,\n",
    "    'test_images': test_images,\n",
    "    'test_labels': test_labels\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 序列化保存在文件\n",
    "with open('../../common/xo_dataset.bin', 'wb') as f:\n",
    "    pickle.dump(dataset, f)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 导出部分测试图片"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2\n",
    "#导出测试数据集\n",
    "for i in range(20):\n",
    "    cv2.imwrite('../../common/test_dataset/test_{}.png'.format(i), test_images[i])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
